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(54) Barcode system 

(57) A barcode system (1 0) generates and decodes 
a barcode with visual significance. The barcode system 
(10) includes an encoding module (14) for receiving a 
message (18) and a logo (16) and generating a visually 
slgnifk:ant barcode (20) based on these inputs. A de> 



coding module (24) is also provided for receiving an ac- 
quired version (48) of a visually significant barcode that 
may have been degraded in a typical document han- 
dling channel (e.g., through the printing, scanning, or 
copying process) and for recovering the message (60) 
specified by the visually significant barcode. 
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Description 

[0001] The present invention relates generally to bar 
codes and more specifically to a method and apparatus 
for generating and decoding a visually significant bar- 5 
code. 

[0002] Barcodes are infomiation carrying graphical 
pattems designed for easy and reliable automatic re- 
trieval. The most common barcodes are known as one- 
dimensional barcodes, Th ese graphical pattems vary In io 
a single dimension (e.g. the horizontal dimension), and 
are constant in the other dimension (e.g.. the vertical 
dimension). One-dimensional barcodes are employed 
in low information content applications Wke product in- 
dex registry (e.g. automatic price tagging and inventory 15 
management), or serial number registry (e.g. test-tube 
tagging in automated medical tests). Common exam- 
ples of one-dimensional barcodes are those barcodes 
that are affixed or printed on the packages of items pur- 
chased at a supermaricet or department store. These 20 
barcodes typically can only encode limited information, 
such as the price of the item and the nrmnufac^urer. The 
items having the barcodes are scanned at a checkout 
counter to facilitate the tallying up of a total receipt. 
[0003] In order to convey more infonrtation on the 25 
same surface area, two^imensionafbarcodesviere de- 
veloped, Two-dimenshnsJ barcodes Involve intricate 
pattems that vary in both the horizontal and the vertk:al 
dimensions. Two-dimensional barcodes are used in ap- 
plications that require more infomnation contents. For so 
example, two-dimensional barcodes can be used to en- 
code mail addresses for automated mail reading and 
distribution systems. Mail canler companies can use the 
two-dimensional bar code on shipping packages to en- 
code shipper infomriation, recipient infomnation, tracking 35 
infonmation, etc. In another example, two-dimensional 
barcodes can be used to encode the compressed con- 
tent of a printed page to avoid the need for optical char- 
acter recognition at the receiving end. 
[0004] U.S. Pat. Nos. 5.060,980, 5,486,686, and 40 
5,459,307 illustrate an exemplary 2D barcode system. 
This system utilizes short black bars that have a forward 
orientation or a backward orientatbn (e.g., bars that are 
oriented at either 45% or 135% with respect to a refer- 
ence) to render the barcode. The two possible orienta- 45 
tions of the bar allow information (e.g., 1 or 0) to be en- 
coded therein. This system offers a limited improvennent 
over the traditional 2D barcode in that the appearance 
of these short black bars while still distracting to the hu- 
man visual system is less so than the traditional 2D bar- so 
code. Consequently, it is asserted by these patents that 
a barcode utilizing these bars can be placed in the back- 
ground of a document wfthout excessively distracting a 
person reading such a document. 

[0005] Unfortunately, the unsightly appearance of 2D S5 
barcodes relegatesthese bar codes to impersonal conrv 
merdal and industrial setting regardless of whether the 
barcode less distracting and can be incorporated into 



the background of a document. For example, in nr\any 
consunrter applteations, the cun-ent visually meaning- 
less barcode pattems are prohibitive, since users are 
more likely to decline the benefits of the application rath- 
er than to include a "barcode" on their letterhead. 
[0006] Another challenge in the design and use of 2D 
barcodes is the ability to produce barcodes by using of- 
fice equipment (e.g., printers) that can be subsequently 
read back also using offk:e equipment (e.g., scanners). 
A problem is printer and scanner distortk)ns that can 
vary across different types and manufacturers of the of- 
fice equipment. 

[0007] The present invention seeks to provide an im- 
proved barcode system. 

[0008] According to an aspect of the present inven- 
tion, there is provided a visually signifbant barcode sys- 
tem as specified in claim 1 . 

[0009] According to another aspect of the present in- 
vention, there is provided a method of generating a vis- 
ually significant barcode as specified in claim 6. 
[001 0] According to another aspect of the present in- 
vention, there is provided a method of decoding a visu- 
ally significant barcode as specified in claim 9. 
[001 1 ] The pref en-ed embodiment can provide a bar- 
code which can be reliably generated and scanned re- 
gardless of the type of machine and the manufacturer 
of the machine. The barcode can be robust to photocop- 
ying (i.e., the barcode can be scanned and infonnation 
embedded therein decoded from a copy of the originally 
rendered barcode) and common ofTice document deg- 
radation. 

[0012] The preferred embodiments can also provide 
a visually significant 2D bar code that is robust to dis- 
tortions in the paper path and that overcomes the dis- 
advantages set forth previously. 
[0O13] The preferred barcode pattern with visual sig- 
nificance can be printed and read by standard offtee 
equipment. 

[0014] The preferred system includes an encoding 
module for receiving a message and a logo and gener- 
ating a visually significant barcode based on these in- 
puts. A decoding module is also provided for receiving 
an acquired verston of a visually significant barcode that 
may have been degraded in a typical document han- 
dling channel (e.g., through the printing, scanning, or 
copying process) and for recovering the message spec- 
ified by the visually significant barcode. 
IPOIS] An embodiment of the present invention is de- 
scribed below, by way of example only, with reference 
to the accompanying drawings, in which: 

FIG. 1 is a block diagram illustrating an office envi- 
ronment in which one embodiment of the visually 
significantbarcode system that does not accept us- 
er input can be inrtplemented. 

FIG. 2 is a block diagram illustrating an offk» envi- 
ronment in which an alternative embodintent of the 
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visually significant barcode system that accepts us- 
er input can be implemented. 

FIG. 3 illustrates a logo of a picture without any user 
input feature and a logo of a picture having a user s 
input feature. 

FIG. 4A is a block diagram illustrating an office ma- 
chine In which the visually significant bar code sys- 
tems of FIG. 1 or FIG. 2 can be implemented. FIG. io 
4B is a block diagram illustrating a computer-read- 
able nrtedium in whtoh the visually signiftoant bar 
code systems of FIG. 1 or FIG. 2 can be implement- 
ed. 

15 

FIG. 5 Is a block diagram illustrating in greater detail 
the encoding modules and the decoding modules 
ofFIG. 1 orFIG. 2. 

FIG. 6 illustrates a method of encoding a message 20 
into a visually significant barcode according to one 
embodiment of the present invention. 

FIG. 7A illustrates an exemplary implementation of 
the encoding step of FIG. 6 in accordance with one 25 
embodiment of the present invention. 

FIG. 7B illustrates pattern matnces according to 
one embodinnent of the present invention that can 
be used in the encoding scheme of FIG. 7A. 30 

FIG. B illustrates a method of decoding a visually 
significant bar code according to one embodiment 
of the present invention. 

35 

FIG. 9 illustrates an exemplary logo image and a 
corresponding visually signifteant barcode. 

FIG. 10 illustrates a zigzag scan that can be used 
in the decoding method of FIG. 6 to locate the bar- 40 
code zone. 

Visually Significant Bar Code System 1 0 

[0016] FIG. 1 Is a block diagram illustrating an office 45 
environment in which one embodiment of the visually 
significant bar code system (VSBCS) 10 that does not 
accept user input can be implemented. The visually sig- 
nificant bar code system (VSBCS) 10 includes an en- 
coding module 14 for generating a visually significant so 
bar code (VSBC) 20 by utilizing a rendering technique 
and a decoding module 24 for decoding an acquired ver- 
sion 48 of a hard copy (e.g., a first hard copy 38, a copy 
54 of the first hard copy 38, or a second hard copy 56) 
that includes a visually significant bar code (VSBC) and S5 
for recovering a message 60 that is specified in the ac- 
quired version 48. The VSBC 20 can be rendered by 
utilizing rendering techniques that are well-known to 



tiiose of ordinary skill in the art. In one embodiment, the 
VSBC 20 is rendered by utilizing a halftone algorithm to 
generate a halftoned VSBC 20. 
[0017] The system 1 0 can communicate with a driving 
program 15 for providing the logo 1 6 and the message 
18 and also communk^te with a driven program 70 for 
utilizing the recovered message 48 for a partbular ap- 
plication. 

[0018] The encoding module 1 4 Includes a first input 
for receiving a logo 16, a second input for receiving a 
message 18. and an output for generating the visually 
significant bar code (VSBC) 20. The message 18 can 
be any information ttiat a system designer specifies. 
[0019] Logo 1 6 can be any graphical pattem that has 
visual significance, which is in sharp contrast to prior 2D 
bar codes that do not make sense as graphical entities. 
For example, logo 16 can Include, but is not limited to, 
an actual logo, graphics, a picture, text, a text box, an 
Image, or any other pattern that has visual significance. 
Exemplary logos can include a company logo, an appli- 
cation logo, or action logo. Exemplary graphics can be 
frames, button marks, or background. An example of 
text Is a text box having ttie words This box contains 
important data!" The VSBC can be a bar code pattem 
ttiat is embedded in the graphical design of text, pic- 
tures, images, borders, or the background of a docu- 
ment. 

[0020] The VSBC pattem may be, but is not limited 
to, a binary Image (e.g., a black and white dot pattem), 
a multi-level image (e.g., a gray-level image), or a mul- 
tilevel color Image that is generated by encoding nnodule 
1 4. The encoding module 1 4 is described in greater de- 
tail hereinafter with reference to FIG. 5. 
p)021 ] The office environment typically includes offbe 
equipment, such as a personal computer (PC) (not 
shown), a printer 30 having a print engine 34, a scanner 
40 having a scan engine 44, a facsimile machine (not 
shown), and a copier 50 for making copies of documents 
(e.g., copy 54). The print engine 34 receives images (e. 
g., halftoned images) from sources, such as the encode 
module 14, and renders these images to create hard 
copies, such as hard copy 38 with a first VSBC (LI ). The 
scan engine 44 receives a hard copy (e.g., a hard copy 
38 or 54 witfi a first VSBC (LI) or a second hard copy 
56 with a second VSBC (L2)), generates an acquired 
version 48 (e.g. a scanned version) of the received hard 
copy, and provides the acquired version 48 to applica- 
tions, such as decoding module 24. ft is noted that the 
scanner 40 can be replaced with a digital camera for 
acquiring a version of the visually significant barcode. 
Furthermore, the scan engine 44 can be any acquisition 
engine for capturing the VSBC and providing an ac- 
quired version (e.g., a digital representation) of the VS- 
BC. The operation and constmction of ttiese office ma- 
chines and their respective components are generally 
well known by those of ordinary skill in the art and are 
not described herein. 

[0022] In a typcal document-handling path, the VSBC 
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along with the remainder of the docunDent is provided to 
a printer, such as printer 30. The printer 30 then utilizes 
the print engine 34 to render a first hard copy 38 that 
includes a first VSBC (LI). The hard copy 38 may then 
be provided directly to a scanner, such as scanner 44, 
to generate a scanned version 48. Altematively, the hard 
copy 38 can first be copied once or several times utiliz- 
ing a copier, such as the copier 50. Thereafter, the cop- 
ied version (e.g., copy 54) can be provided to the scan- 
ner 44 to generate the scanned version 48. It is noted 
that a second hard copy (e.g., hard copy 56) can be gen- 
erated (e.g., printed or copied) by utilizing a printer (not 
shown) different from printer 30 that is part of a separate 
office. Also, it is noted that the logo 16 can vary from 
application to application (e.g., first VSBC (L1) or sec- 
ond VSBC (L2)) and can be specified by a system de- 
signer 

[0023] The decoding module 24 receives the acquired 
version 48 and recovers the message 60 encoded in the 
acquired version 48. For example, if the acquired ver- 
sion conresponds to the first hard copy 38 or to the copy 
54, then the message in the first VSBC (L1 ) is recovered 
as the message 60. Altematively, if the acquired version 
conresponds to the second hard copy 56 or photocopies 
thereof, then the message in the second VSBC (L2) is 
recovered as the message 60. The message 60 is then 
provided to a driven application 70 that perfomns one or 
more pre-determined functions or tasks, examples of 
which are described in greater detail hereinafter with ref- 
erence to FIG. 3. 

[0024] FIG. 2 is a block diagram illustrating an office 
environment in whbh an alternative embodiment of the 
visually signifk^ant barcode system 1 0A that accepts us- 
er input can be implemented. This visually significant 
bar code system 1 0B is similar to the system 1 0A Illus- 
trated in FIG. 1 , except that this system 10A is config- 
ured to accept user input. A user can input Infonmation 
to the first hard copy 38, the copy 54, or the second hard 
copy 56 in the fomn of marics made by using a writing 
utensil (e.g., a pencil or pen) as will be explained in 
greater detail hereinafter with reference to FIG. 3. The 
system 10A includes a user input decoding module 26 
for decoding the user input, such as (1) user input 64 
that may be provkied to the hard copy 38; or (2) user 
input 65 that may be provided to the copy 54 or the sec- 
ond hard copy 56. it is noted that the user input decoding 
module 26 may be integrated with the decoding module 
24 or be separate from the decoding module 24 as 
shown. 

[0025] In this embodiment, the decoded user input 66 
and the message 60 specified by the acquired image 48 
are provided to a decision module 68 that generates one 
or more signals based on the user input 66 and the mes- 
sage 60. These signals can specify or select from one 
of several options provided by the message. These sig- 
nals are then provided to one or more driven programs 
or applications 70 that perfonn pre-determined tasks or 
functions, launch other applk^ons, or control other ap- 



plications to perf omfi the detemnined tasks. Examples of 
such task are described hereinafter with reference to 
FIG. 3. 

[0026] It is noted that the driving applk»tion 15 and 

5 the driven application 70 can be the same application 
or different applications. Furthemnore, the driving appli- 
cation 15 and the driven application 70 can be incorpo- 
rated into the visually significant bar code system (VS- 
BCS) 10 or be separate from the VSBC system 10. 

10 [0027] FIG. 3 illustrates a logo 304 of a picture 304A 
without any user input feature and a logo 304B of a pic- 
ture having a user input feature. A document 300 (e.g., 
documents 38, 54, or 56) includes a non-logo portion 
302 (e.g., text, images, pbtures etc.) and a logo portion 

IS 304 (e.g., first VSBC (LI ) or second VSBC (L2)) that is 
can be a VSBC 20. It is noted that the VSBC 20 is pro- 
vided along with the non-k>go portion so that a printer 
(e.g., the printer 30) can render the entire document 
300. Since the rendering and decoding non-logo por- 

20 tions of documents are well known by those of ordinary 
skill in the art, these operations are not described herein. 
[0028] The logo 304 can be a ptoture 304A without 
any user input feature or a pcture 304B with a user input 
feature. In either case, the picture (304A or 304B) in- 

2s dudes an encoded message 308 that can be, but is not 
limited to, an Intemet address (e.g., a Worid Wide Web 
address), an electronic mail address, authentication in- 
fomiation (e.g., a digital signature orthe author or send- 
er of the document), a file name and file location, a fac^ 

30 simile number for "fax-back" applications, or a combina- 
tion thereof. 

[0029] One embodiment provides a VSBC to increase 
the value of documents and to create "living docu- 
ments". Living documents are those documents that 

35 have more information (e.g., encoded messages) than 
meets the eye or is apparent in viewing the documents. 
Living documents have encoded messages in the form 
of VSBCs that can be used to dynamteally retrieve ad- 
ditional Information (e.g., updated versions of the doc- 

40 ument), to reduce human intervention, to increase con- 
venience, to improve effbiency in the office, to increase 
security in the distribution of documents, to increase 
productivity and to facilitate interaction and communica- 
tion between people in the office. 

45 [0030] There are numerous applications that use one 
or more of the types of messages, described above, and 
encodes these types of messages into documents. It is 
to be appreciated that there are many other types of 
messages or information that may be encoded into doc- 

50 uments and numerous applications related thereto. The 
cornerstone of these applkjations Is the VSBCS de- 
scribed herein. 

[0031] For example, one application is to use the VS- 
BC to store the file name and the file location (e.g. , path 
55 to access the document on a sen/er, Web site, etc.). 
When a document having a VSBC that contains file and 
file location infonfnation is scanned into or otherwise ac- 
quired by a personal computer, a Most-Recent Docu- 
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ment Application can utilize this infomiation to access 
the nfK>st recent or updated electronic version of the doc- 
ument and launch the document by using a word 
processing application. 

[0032] As can t)e appreciated, the system can provide 
significant improvements over the prior art approach 
that would attempt to convert the scanned version into 
a text-editable version via some type of OCR. The OCR 
approach Is prone to many errors and mistakes in rec- 
ognition. In addition, regardless how accurate the OCR 
utilized, only the version as shown on the physical doc- 
ument can be accessed. If that document has been sub- 
sequently revised, there is no mechanism that would al- 
low for the retrieval of the most up-to-date version. This 
application retrieves the most-current electronic version 
even if the document has poor quality text that is inad- 
equate for recognition by prior art OCR methods be- 
cause the file name and file path are encoded in the VS- 
BC. 

[0033] Similarty, the encoded name and location of 
the original electronic version of the document, allows 
a copier to utilize the original document to be utilized as 
a copy source instead of having to rely on the document 
in-hand with the VSBC, which may have been severely 
degraded in the typical paper handling path. This is an 
example of a Copy-from-Electronic- Version Application. 
[0034] A security application can utilize authentication 
information encoded in the VSBC for authentication pur- 
poses or other secure conrvnunication functions. For ex- 
ample, a digital signature can be encoded into the VSBC 
so that the source of the document can be authenticat- 
ed. 

[0035] In another example, the facsimile number is 
encoded in the VSBC so that an Automatic Fax-Back 
Application can utilize the encoded facsimile number to 
automatically dial the number when the document is fed 
into a facsimile machine without user intervention. Al- 
ternatively, an e-mail address may be encoded in the 
VSBC so that an Automata Electronic-Mail Response 
Application can utilize the encoded electronic mail ad- 
dress to automatically send nrrare infomnation (e.g., a re- 
sponse) via e-mail to the author of the document. 
[0036] The pk^ture 304B with a user input feature can 
include one or more user input forms. For example, a 
Check Box Form 314 can be provkledfora userto select 
from one of several options by simply checking the ap- 
propriate box. The text nnessage box "Check Appropri- 
ate Box" 315 can be a visually significant barcode. In 
another example, a Circle Selection Fonm 318 can be 
provided for a userto select from one of several options 
by simply circling the appropriate selection. The text 
message box "Circle Desired Option" 319 can be a vis- 
ually significant barcode. In yet another example, a Rll- 
In Form 324 can be provided for a user to fill in informa- 
tion requested by the fomn. The text nrtessage box "nil 
In" 325 can be a visually significant barcode. 
[0037] Each of these forms request user input 330, 
such as a check mark, a circle, or text input. The user 



input 330 can then be decoded by the user input decode 
module 26 to recover the user input. Thereafter, the de- 
cision module 68 utilizes the user Input in conjunction 
with the recovered message to select one of several 

5 available functions or tasks, and the driven application 
70 for perfomning the selected tasks can be launched. 
For example, the user Input can be utilized to arbitrate 
between different messages encoded in the different 
visually signiffcant text boxes. In the automatic e-mail 

10 response application, the user input and the different 
visually signif k:ant text boxes can be utilized to (1 ) reply 
only to the sender or (2) to reply to all recipients including 
the sender. 

[0038] The VSBC system 1 0 or 1 0A can be embodied 
IS on a media that is incorporated in offrce equipment (e. 
g., a personal computer (PC), printer, facsimile ma- 
chine, scanner, copier, alUn-one machine, etc.) or sep- 
arate from offce equipment. When incorporated in offfce 
equipment, the media, having the VSBC system 10 or 
20 1 0A embodied therein, can be in the form of a memory 
(e.g., nandom access memory (F^AM), read only mem- 
ory (ROM), etc.). When incorporated separate from the 
office equipment the media, having VSBC system 10 
or 10A embodied therein, can be in the fomn of a com- 
25 puter-readable medium, such as a floppy disk, compact 
disc (CD), etc. 

[0039] FIG. 4A is a block diagram illustrating an office 
machine 400 in whfch the VSBC system 1 0 or 1 0A can 
be implemented. The office machine 400 includes a 

30 memory 410 in which the VSBCS 10 or IDA of the 
present invention can reside. The offk» machine 400 
may be a personal computer (PC). If this is the case, the 
PC can be coupled to a printer, scanner, an all-in-one 
machine or other peripheral machine. It is noted that the 

35 VSBCS 1 0 or 1 0A may reside in the PC in the fomi of a 
driver program associated with one of these peripheral 
machines or can be stored in a memory that is located 
in the peripheral machines. 

[0040] Altematively, the offtoe machine can be an all- 

^ in-one offtee machine (e.g., an integrated scanner, print- 
er, and facsimile machine) or any rendering machine. 
[0041] FIG. 4B is a block diagram illustrating a com- 
puter-readable medium 420 in whnh the VSBC system 
1 0 or 1 0A of the present invention can be implemented. 

^ The computer-readable medium 420 may be a floppy 
disk 440, a compact disc (CD) 430, or other computer- 
readable medium, etc. As can be appreciated, the VS- 
BC system 10 or 10A can be implemented in a variety 
of other ways and in a variety of other components that 

so are commonly found in an office environment. 

[0042] FIG. 5 is a block diagram illustrating in greater 
detail the encoding module 14 and the decoding module 
24 of FIG. 1 and FIG. 2. The encoding module 14 in- 
cludes an encoder 500 for receiving a message 1 8 and 

ss generating an encoded message 504 based on the 
message 18. Encoding module 14 also includes a 
graphic modulatton unit 510 for receiving the logo 16 
and the encoded message 504 and based thereon, gen- 
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erates the VSBC 20. The VSBC 20 can be a specific 
pattern. In the channel 514, the VSBC 20 can be ren- 
dered directly on a document or incorporated Into a doc- 
ument and rendered thereon. 
[0043] Preferably, the visually significant two-dimen- 
sional barcodes are graphical patterns composed, usu- 
ally though not necessarily, of dots. The visually signif- 
icant bar codes are rendered, usually though not nec- 
essarily, using two-toned dots (e.g. black dots on a white 
background), and occupy, usually though not necessar- 
ily, a rectangular area. The visually significant bar codes 
can incorporate various registration and fiducial marks, 
whbh are generally well known to those of ordinary skill 
in the art, for enabling automated klentification and ac- 
curate registration of the bar code, which may be read 
In arbitrary orientations. In addition, the visually signifi- 
cant bar code system can employ various error correct- 
ing codes, which are generally well known to those of 
ordinary skill In the art, for reliable automated retrieval 
of the bar codes. 

[0044] The steps earned out by encoding module 14 
to process the message 1 8 and the logo 1 6 to generate 
the VSBC 20 are described in greater detail hereinafter 
with reference to FIG. 6.. 

[0045] Channel 514 represents a typical office docu- 
ment-handling path for hard copies by standard office 
equipment. For example, a hard copy can be printed, 
photocopied, and scanned. It is understood that one or 
more of these standard off tee handling steps can be car- 
ried out by multi-functional equipment. Since the bar- 
code pattem is subject to noise and office document 
degradation (e.g., folds, stains, mari(s, and staples) in 
the channel 514, a standard cleaning process to ad- 
dress the noise and degradation of barcode pattem can 
be employed. 

[0046] The steps canied out by the decoding module 
24 to recover the message 60 from the acquired image 
48 are described in greater detail hereinafter with refer- 
ence to FIG. 8. 

Generating the VSBC 20 

[0047] FIG. 6 illustrates a method of encoding a mes- 
sage 1 8 into a visually signifteant barcode 20 according 
to one embodiment of the present invention. In step 600, 
the message 1 8 and the logo 1 6 are received. The mes- 
sage 1 8 can also be compressed by utilizing well known 
compression technkjues in this step. For certain appli- 
catbns. such as universal resource locators (URLs), a 
tailored or customized compresston scheme may be 
preferable. 

[0048] In step 610, the compressed message is en- 
coded by utilizing an error conrectton code with an output 
alphabet L to generate a message encoded in a se- 
quence of O symt>ols. This encoding step provides ro- 
bustness to en-ors in the channel 514. For example, a 
standard 16 31 bit BCH code that corrects for three 
errors, where L = 4 (2 bits) can be used. 



[0049] In step 620, the logo 16, which can be an M x 
N pixel image, is partitioned in a plurality of image ma- 
trices. In one example, K is equal to two (2), and M and 
N (i.e., the size of the logo 1 6) are even values (e.g., N 
5 =80 and M = 40), The values of K, M and N can be 
modified and predetermined by a system designer to 
suit a particular application. 

[0050] In step 630, an image area con^espondlng to R 
logo matrices Is used for pre-defined fiducial marics. For 

10 example, the four comers of the image can be reserved 
for fiducial marks. In each comer, an area of 4 x 4 pixels 
(2X2 matrices) can be designated for fiducial marics. 
The fiducial marks can be the whole area rendered 
white, except for an isolated location In each comer that 

IS is set apart or separate from adjacent matrices, which 
is rendered black. The upper-left fiducial marie, for ex- 
ample, can be all white except for the upper-left pixel, 
which is rendered black. It is preferable that the pattem 
for the fiducial marie be as robust to channel noise as 

20 possible. Since dots may be blun-ed or move relative to 
each other due to noise in the channel 514, this pattem 
ensures that the black fkiucial dots do not merge with 
neighboring dots, and instead stand out clearly on a 
white background. Various otherfiducial pattems can be 

25 utilized depending on the particular types of noise in a 
particular channel of interest. 

[0051] In step 640, the remaining bgo matrices (I.e., 
P=Mx Nf{K xK}'R logo matrices) are converted to K 
xKbinary barcode matrices by utilizing one of a prede- 

30 fined set of L distinct maps (e.g.. halftoning algorithms) 
based on a comespondlng symbol in the coded mes- 
sage. In one example, there are 784. remaining image- 
matrices (based on N=BO and M =40) that can be or- 
dered in raster scan. The 784 image matrices can ac- 

55 commodate slightiy more than 25 batches of 31 matri- 
ces (O = 25 x 31 = 775). It is important to note that P 
should be greater than Q. Each batch codes two batch- 
es of 16 input bits: (1) one batch for the BCH MS bits, 
and (2) the other batch for the BCH LS bits. Thus, for 

40 the parameters specified above, the barcode can en- 
code 25 X 16 X 2 = 800 bits of Infonnatlon. 
[0052] FIG. 7A illustrates an exemplary implementa- 
tion of encoding step 640 of FIG. 6 in accordance with 
one embodiment of the present invention. The logo 16 

45 includes a plurality of pixels having one of two different 
gray levels: 25% and 75%. For example, pixels 704, 706 
have a 25% gray level, and pixel 708 has a 75% gray 
level. 

[0053] Refening to FIG. 7B, for those pixels in the logo 
50 1 6 that have a 25% gray level, one of the four bar code 
matrices In the first row is used to render the based on 
the data to be encoded. For example. If the data to be 
encoded is "IT, pixel 704 is rendered by utilizing the 
bar code matrix 71 0. Similarly, if the data to be encoded 
55 is "01", pixel 706 is rendered by utilizing the bar code 
matrix 730. SImilariy, for those pixels in the logo 1 6 that 
have a 75% gray level, one of the four bar code matrices 
in the second row is used to render the based on the 
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data to be encoded. For example, if the data to be en- 
coded is '^0". pixel 708 is rendered by utilizing the bar 
code matrix 720. 

[0054] Although this umplementation utilizes two gray 
levels, 25% and 75%, it is noted that the system can be 
extended to Include multiple levels can t>e utilized to suit 
a particular application. In addition, It can be extended 
to pixels of different colors. 

[0055] Altematively, the remaining image-matrfces 
can be ordered with Interleaving methods that are 
known in the art, to provide robustness to burst-type 
degradations expected from stains, marks, or system- 
atic printer/scanner distortions. The K x K binary bar- 
code matrices can then be sent to a printer for printing 
onto a hard copy, such as a piece of paper. 
[0056] There are many rendering methodologies from 
whtoh one can choose the L distinct maps. For example, 
when halftoning algorithms are selected as the L distinct 
maps, there are many type of halftone methodologies 
one can choose from. In an exemplary implementation 
a fixed-halftone-pattem halftoning method is utilized. 
This method specifies that the image be a 2-tone image. 
If black is 0, and white is 1, the bright tone b, and the 
daric tone d, are such that d=1-b. Also, the L halftoning 
algorithms correspond to L distinct Kx K pattem-matri- 
ces, where each pattern matrix contains 6 • K x /Cblack 
dots on white background. However, it is noted that oth- 
er well-known halftoning methods, such as cluster dith- 
ering, disperse dithering (e.g., blue noise), and enror dif- 
fusion methods can be utilized. When an error diffusion 
algorithm Is employed, one can select from nnany differ- 
ent methods to diffuse the error Similarly, when a dis- 
perse dithering algorithm is utilized, one can select from 
many methods to define the dither matrices (also known 
as screens) of various sizes. 

[0057] Given a logo-matrix, and the selected pattem- 
matrix, the resulting barcode-matrix contains the pattern 
matrix values in the places conrespondlng to the bright 
pixels In the logo-matrix, and their complementary oth- 
erwise. 

[0058] In this embodiment, the dots are preferably 
rendered as square dots at 120 dots-per-inch (dpi). It is 
noted that larger dots are more robust to channel deg- 
radations, but smaller dots enable more informatkm on 
the same area of the paper. The dots can be rendered 
at a dpi higher than 120dpi as long as acceptable error 
resilience can be assured. 

[0059] It is noted that the size of the dots can tie varied 
according to intensity. An example of varying the size of 
dots based on intensity is to make the black dots slightly 
larger than the white dots. It is also noted that the size 
of the dots can be varied according to neighborhood. An 
example of varying the size of dots based on neighbor- 
hood is to make minority colored dots larger. 
[0060] FIG. 9 illustrates an exemplary logo 16 and 
message 1 8 that are used to generate a con-esponding 
VSBC 20 by the disclosed encoding method. 



Decoding the Acquired Barcode Image 48 

[0061] The image 48 acquired by the scanner 40 and 
introduced to the decoding module 24 is a degraded ver- 
5 sion of the original barcode-image. These degradattons 
are attributed to the channel 514, namely, the printing 
and scanning processes, and potential office type deg- 
radations, such as copying, stains, folds, staples, and 
marks. 

10 [0062] FIG. 8 illustrates a method of decoding a visu- 
ally significant barcode according to one embodiment of 
the present invention. In step 800, the acquired image 
is received, for example, from the scanner In step 810, 
the barcode image is located in the received image. For 

15 example, the received image can be a whole page of 
information, whereas the barcode image can be limited 
to a barcode zone, which may be a portion of the whole 
page. In one embodiment, the barcodes are located 
roughly at a same predetenmined location in the ac- 

20 quired image and are surrounded by white pixels. This 
predetermined location is refen-ed to herein as the bar- 
code-zone. Alternatively, the barcodes can be disposed 
in varying locations on a page, as background, or as a 
frame for the page. When barcodes are disposed in 

25 such a manner, a barcode locating procedure can be 
utilized to locate the barcode image on the page. 
[0063] In step 820, the fiducial marics are detected. 
The configuration of these fiducial marks indicates the 
type of global, deformatton introduced by the channel. 

30 In one embodiment, the fiducial marics are detected by 
using a zigzag scan. 

[0064] FIG. 1 0 Illustrates a zigzag scan 1 000 that can 
be used in step 820 to detect the fiducial marics. The 
zigzag scan 1000 begins at the upper-left comer 1010. 

35 The first dari< pixel 1 020 of each scan (requires a thresh- 
old) is detected and considered to be a part of the cor- 
responding fiducial pixel. The first daric pixel 1 020 is also 
used as an anchor pixel for that mark (red dot in Figure 
5). Next, a flood-fiil algorithm, whk:h is known in the art, 
locates all the daric pixels 1030 connected to these an- 
chors. The centers 1040 of the fiducial marics are then 
(X)mputed as the average (center of mass) of the pixels 
of each marie. An advantage of this zigzag scan is that 
it enables a robust detection of the anchor points even 

45 in the presence of significant rotations. Thfe scan can 
be repeated at each comer of the barccxJe. 
[0065] In step 830, viewing transfomnatlon distortions 
(e.g., translation, rotation, and affine) are conrected. By 
setting the center of the coordinate system to be ten pix- 

50 els above and ten pixels to the left of the center of the 
upper-left fiducial marie, translation problems can be 
eliminated. The relative rotation of the center of the up- 
per-right fiducial marie is then detemnined, and the image 
is rotated back accordingly. 

55 [0066] Next, the skew factor, which is the horizontal 
translation of the centers of lower marics relative to the 
horizontal translation of the centers of the upper mark, 
is (ietenmined. If a skew is detected, the image is cot" 
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rected for the detected skew. These transformation pro- 
cedures can be implemented with standard procedures 
known In the computer vision and image processing ar- 
eas. 

[0067] In one embodiment, bilinear interpolation is uti- 
lized although other Interpolators can be used. It is noted 
that a single aff Ine transformation can be utilized to cor- 
rect for both the rotation and the skew. 
[0068] The four fiducial marks enable correctton of 
global transformations with up to eight degrees of free- 
dom. In this embodiment there are four degrees of free- 
dom (two - translation, one - rotation, and one- skew). 
[0069] In step 840, the acquired barcode Image Is par- 
titioned into a plurality of sub-images that can be ar- 
ranged in a rectangular array, where each sub-image 
corresponds to a single barcode matrix. Step 830 pro- 
vides a rectangular image. In one embodiment, the step 
of partitioning the acquired barcode image involves the 
steps of measuring the image and slicing the image Into 
rectangular sub-images. In our case M//C= 20, and N 
/K= 40 so each rectangular sub-image had the dimen- 
sions of 20 X 40. 

[0070] In step 850, each sub-image is compared 
against the set of L possible barcode-matrices (i.e., the 
outputs of the L distinct maps (e.g., halftoning algo- 
rithms) given the corresponding image), When the half- 
tone pattems of FIG. 7B are used, and the logo design 
is required to have a constant brightness in every logo- 
matrix, four vector products can be applied to every sub- 
Image. The four vectors are Gaussian profiles that are 
each centered at the center of a quadrant of the sub- 
image. For dark sub-images, the pattern of FIG. 7B cor- 
responds to the quadrant having the maximal value, and 
for bright sub-images, the pattem conresponds to the 
quadrant having the minimal value. This approach Is al- 
so known as the matched filter approach in the image 
processing arts. 

[0071] In step 860, a best match is selected to repre- 
sent the sub-image In a sequence of P synnbols over 
{1 ,2,... L}. It is noted that any maximum-likelihood-type 
of detector or any other match estimator can be utilized 
to determine whbh of the L possible maps (e.g., half- 
tones) is the most likely to have produced the corre- 
sponding sub-image. Preferably, the best match is per- 
fomned on a group of sub-images. 
[0072] In step 870, the (possibly enroneous) se- 
quence that was originally coded with an error-correct- 
ing code is decoded, thereby eliminating the effect of 
possible errors due to the degradations of the channel. 
The corresponding standard BCH error comection may 
be periomned followed by the appropriate re-ordering of 
the bits. 

[0073] In step 880, the sequence is decompressed to 
give the original message. 

[0074] The present Invention provides a barcode pat- 
tem with visual significance, thereby irtcreasing its ac- 
ceptance and use in the business environment and even 
the home environment Whereas the prior art bar codes 



were limited to Impersonal commercial and industrial 
applications, the barcode pattem with visual signifi- 
cance changes the sentimental value attached the use 
of barcodes in more personal contexts from unfavorable 

5 to favorable. 

[0075] In addition , different applications can easily in- 
corporate the visually significant barcode pattern and 
different variations thereof (e.g., different logos or other 
graphk^s), thereby enabling a user to customize or per- 

10 sonalize the barcode pattem. 

[0076] Moreover, the system utilizes techniques, such 
as error con-ection and the use of fiducial marks, to make 
the visually significant barcode pattem robust to imper- 
fections and noise in the hardcopy-handling channel. 

IS For example, the barcode pattem is robust to consecu- 
tive photocopying by common office copiers (e.g., ana- 
log and digital copiers). Also, the barcode pattem is re- 
silient to common offk^e document degradations, such 
as folds, stains, marks, and staples. 

20 [0077] Furthermore, the visually signifteant barcode 
pattem can be printed by and read by standard offbe 
equipment, such as printers, scanners, copiers, and 
multi-functional equipment. 

[0078] The disclosures in United States patent appli- 
25 cation no. 09/579,070, from which this application 
claims priority, and in the abstract accompanying this 
application are incorporated herein by reference. 

30 Claims 

1 . A visually signif k:ant barcode system (1 0) including: 

an encoding module (1 4) for receiving a nnes^ 
sage (18) and a logo (16) and based thereon for 
35 generating a visually signif bant barcode (20) hav- 
ing the message encoded therein. 

2. A system as in claim 1 , including: 

a print engine (34) coupled to the encoding 
^ module for receiving the visually significant barcode 
and based thereon for rendering a hard copy of the 
visually significant barcode. 

3. A system as in daim 2, wherein the print engine uti- 
^ lizes a halftone algorithm to render the hard copy of 

the visually sign'ifk^ant barcode. 

4. A system as in any preceding claim, including: 

a decoding module (24) for receiving an ac- 
50 quired version (48) of the visually significant bar- 
code and based thereon for recovering a message 
(60) encoded therein. 

5. A system as in claim 4, wherein the encoding mod- 
55 ule and the decoding nradule are embodied on a 

medium. 

6. A system as in claim 5, wherein the media is incor- 
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porated in an office machine in the fomn of a mem- 
ory. 

7. A system as In claim 4, 5 or 6. Including: 

an acquisition engine (44) coupled to the de- 
coding module for receiving a hard copy having a 
visually significant barcode and, based thereon, for 
generating the acquired version of the visually sig- 
nificant barcode. 

8. A method for generating a visually significant bar- 
code comprising: 



receiving an M x N pixel image (600); 

receiving a message having a plurality of fields is 

(600): 

partitioning the M x N pixel image into a plurality 
of K X K image matrices (620); and 
converting the K x K image matrices to K x K 
barcode matrices by utilizing one of a predeter- 20 
mined set of L distinct maps; wherein the se- 
lection of the particular map is based on a cor- 
responding field of the message (640). 

A method of decoding a visually significant barcode 2s 

comprising: 

receiving the barcode image (800); 
partitioning the barcode Image Into a plurality 
of sub-images (840); 30 
comparing each-sub-image with a set of L pos- 
sible barcode matrices (850); and 
decoding a message based on a match estima- 
tion of each sutHmage to each one of the L pos- 
sible barcode matrices in a sequence of P sym- 35 
bols over {1,2, ...L}(860). 
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